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ABSTRACT 

We introduce a new method, combination of random test- 
ing and abstract interpretation, for the analysis of programs 
featuring both probabilistic and non-probabilistic nondeter- 
minism. After introducing "ordinary" testing, we show how 
to combine testing and abstract interpretation and give for- 
mulas linking the precision of the results to the number of 
iterations. We then discuss complexity and optimization is- 
sues and end with some experimental results. 

1 INTRODUCTION 

We introduce a generic method that lifts an ordinary ab- 
stract interpretation scheme to an analyzer yielding upper 
bounds on the probability of certain outcomes, taking into 
account both randomness and ordinary nondeterminism. 

1.1 Motivations 

It is sometimes desirable to estimate the probability of cer- 
tain outcomes of a randomized computation process, such 
as a randomized algorithm or an embedded systems whose 
environment (users, mechanical and electrical parts. . . ) is 
modeled by known random distributions. In this latter case, 
it is particularly important to obtain upper bounds on the 
probability of failure. 

Let us take an example. A copy machine has a comput- 
erized control system that interacts with the user through 

"This work was partially funded by Commissariat a 
FEnergie Atomique under contract 27234/VSF. 



some control panel, drives (servo)motors and receives infor- 
mation from sensors. In some circumstances, the sensors 
can give bad information; for instance, some loose scrap of 
paper might prevent some optical sensor from working cor- 
rectly. It is nevertheless desired that the probability that 
the machine will stop in an undesired state (without having 
returned the original, for instance) is very low given some re- 
alistic rates of failure from the sensors. To make the system 
more reliable, some sensors are redundant and the control- 
ling algorithm tries to act coherently. Since adding sensors 
to the design costs space and hardware, it is interesting to 
evaluate the probabilities of failure even before building a 
prototype. A similar case can be made of industrial systems 
such as nuclear power plants were sensors have a limited life 
time and cannot be expected to be reliable. Sound analysis 
methods are especially needed for that kind of systems as 
safety guidelines are often formulated in terms of maximal 
probabilities of failures |10j . 

1.2 Nondeterminism and Probabilities 

Treating the above problem in an entirely probabilistic fash- 
ion is not entirely satisfactory. While it is possible to model 
the user by properties such as "the probability that the user 
will hit the C key during the transfer of double-sided doc- 
uments is less than this can prevent detecting some 
failures. For instance, if pressing some "unlikely" key combi- 
nation during a certain phase of copying has a good chance of 
preventing correct accounting of the number of copies made, 
certain users might use it to get free copies. This is cer- 
tainly a bug in the system. To account for the behavior of 
inputs that cannot be reliably modeled by random distribu- 
tions (for instance, malicious attacks) we must incorporate 
nondeterminism. 

1.3 Comparison to other works 

An important literature has been published on software test- 
ing [1311191 . . .]; the purpose of testing techniques is to dis- 
cover bugs and even to assert some sort of reliability criterion 
by testing the program on a certain number of cases. Such 
cases are either chosen randomly (random testing) or accord- 
ing to some ad hoc criteria, such as program statement or 
branch coverage (partition testing). Partition-based meth- 
ods can be enhanced by sampling randomly inside the par- 



tition elements. Often, since the actual distribution in pro- 
duction use is unknown, a uniform distribution is assumed. 

In our case, all the results our method gives are relative 
to some fixed, known, distributions driving some inputs. On 
the other hand, we will not have to assume some known 
distribution on the other inputs: they will be treated as 
nondeterministic. We thus avoid all problems pertaining to 
arbitrary choices of partitions or random distributions; our 
method, contrary to most testing methods, is fully mathe- 
matically sound. 

There exists a domain called probabilistic software engi- 
neering |14] also aiming at estimating the safety of soft- 
ware. It is based on statistical studies on syntactic aspects of 
source code, or software engineering practices (programming 
language used, organization of the development teams. . . ), 
trying to estimate number of bugs in software according to 
recorded engineering experience. Our method does not use 
such considerations and bases itself on the actual software 
only. 

Our analysis is based on a semantics equivalent to those 
proposed by Kozen 8,9, 2nd semantics] and Monniaux . 
We proposed a definition of abstract interpretation on prob- 
abilistic programs, using sets of measures, and gave a generic 
construction for abstract domains for the construction of an- 
alyzers. Nevertheless, this construction is rather "algebraic" 
and, contrary to the one explained here, does not make use 
of the well-studied properties of probabilities. 

Ramalingam [15] proposed an abstraction using vectors of 
upper bounds of the probabilities of certain properties, the 
resulting linear system being solved numerically. While his 
approach is sound and effective, it is restricted to programs 
where probabilities are only introduced as constant transi- 
tion probabilities on the control flow graph. Furthermore, 
the class of properties is limited to data-flow analyses. 

Several schemes of guarded logic commands 5^ or refine- 
ment [12] have been introduced. While these systems are 
based on semantics broadly equivalent to ours, they are not 
analysis systems: they require considerable human input and 
are rather formal systems in which to construct derivations 
of properties of programs. 

1.4 Contribution 

We introduce for the first time a method combining statisti- 
cal and static analyses. This method is proven to be math- 
ematically sound. While some other methods have been re- 
cently proposed to statically derive properties of probabilis- 
tic programs in a general purpose programming language 
[11] . ours is to our knowledge the first that makes use of 
statistical convergences. 

1.5 Structure of the paper 

We shall begin by an explanation of ordinary testing and 
its mathematical justification, then explain our "abstract 
Monte-Carlo" method (mathematical bases are given in ap- 
pendix). We shall then give the precise concrete seman- 
tics that an abstract interpreter must use to implement our 
method, first for a block-structured language then for arbi- 
trary control graphs. We shall finish with some early results 
from our implementation. 



2 ABSTRACT MONTE-CARLO: THE 
IDEA 

In this section, we shall explain, in a mathematical fashion, 
how our method works. 

2.1 The Ordinary Monte-Carlo Testing 
Method 

The reader unfamiliar with probability theory is invited to 
consult appendix\A\ 

Let us consider a deterministic program c whose input 
x lies in X and whose output lies in Z. We shall note 
|c] : X i— > Z the semantics of c (so that [c] (a;) is the re- 
sult of the computation of c on the input x). We shall take 
X and Z two measurable spaces and constrain [c] to be 
measurable. These measurability conditions are technical 
and do not actually restrict the scope of programs to con- 
sider [11] , For the sake of simplicity, we shall suppose in this 
sub-section that c always terminates. 

Let us consider W C Z a measurable set of final states 
whose probability we wish to measure when x is a random 
variable whose probability measure is fi. The probability of 
W is therefore ^([cj" 1 (W)). Noting 

Ml) Ji if[c](x)ew 

1 otherwise, 

this probability is the expectation Etw- 

Let us apply the Monte-Carlo method for averages to this 
random variable tw (see appendix [Bj. Etw is then approx- 
imated by n random trials: 


for i = 1 to n do 

x <— random(pi) 

run program c on input x. 

if program run ended in a state in W then 
c <- c+ 1 

end if 
end for 
p <— c/n 

A confidence interval can be supplied, for instance using the 
Chernoff-Hoeffding bound (Inequ. Ill}) : there is at least a 
1 — e probability that the true expectation Etw is less than 

p = p + \J ^ ~ 2*rf ^ (FifS- HI — we shall see the implications in 
terms of complexity of these safety margins in more detail 
in section 3}. 

This method suffers from two drawbacks that make it un- 
suitable in certain cases: 

• It supposes that all inputs to the program are either 
constant or driven according to a known probability 
distribution. In general, this is not the case: some in- 
puts might well be only specified by intervals of possible 
values, without any probability measure. In such cases, 
it is common [13] to assume some kind of distribution 
on the inputs, such as an uniform one for numeric in- 
puts. This might work in some cases, but grossly fail 
in others, since this is mathematically unsound. 

• It supposes that the program terminates every time 
within an acceptable delay. 



We propose a method that overcomes both of these prob- 
lems. 

2.2 Abstract Monte-Carlo 

We shall now consider the case where the inputs of the pro- 
gram are divided in two: those, in X, that follow a random 
distribution /j, and those that simply lie in some set Y. Now 
[c] : X x Y — *■ Z. The probability we are now trying to 
quantify is fi{x € X \ 3y G Y [c] (x,y) £ W}. Some techni- 
cal conditions must be met so that this probability is well- 
defined; namely, the spaces X and Y must be standard Borel 
spaces [7j Def. 12.5] □ Since countable sets, R, products of 
sequences of standard Borel spaces are standard Borel [7] 
§12.B], this restriction does not concern most semantics. 
Noting 



t w {x) 



1 ifByel" Ic] (x,y}eW 
otherwise, 



this probability is the expectation Et w . 

While it would be tempting, we cannot use a straight- 
forward Monte-Carlo method since, in general, tw is not 
computable^ 

Abstract interpretation (see appendix [Cj is a general 
scheme for approximated analyses of safety properties of pro- 
grams. We use an abstract interpreter to compute a function 
Tw '■ X — > {0, 1} testing the following safety property: 

• Tw(x) = means that no value of y £ Y results in 

• Tw(x) — 1 means that some value of y 6 Y may result 
in Icl(x,y) € W. 

This means that for any x, t w (x) < Tw(x). Let us use the 
following algorithm: 


for i = 1 to n do 

x <— random(/i) 
c <— c + Tw (x) 



x Let us suppose X and Y are standard Borel spaces [7J 
§12.B]. X x Y is thus a Polish space fJJ §3. A] so that the 
first projection 7Ti is continuous. Let A — {x £ X | 3y G 
y 14 (x,y) 6 W}; then A = ■Ki(M~ 1 (W)). Since [c] is a 
measurable function and W is a measurable set, [c]~ (W) 
is a Borel subset in the Polish space X x Y. A is therefore 
analytic [7J Def. 14.1]; from Lusin's theorem [7J Th. 21.10], it 
is universally measurable. In particular, it is /^-measurable 
[7J §17. A]. fi(A) is thus well-defined. 

2 Let us take a Turing machine (or program in a Turing- 
complete language) F. There exists an algorithmic transla- 
tion taking F as input and outputting the Turing machine 
F computing the total function tpp, so that 



1 if F terminates in y or less steps on input x 
otherwise. 



Let us take X — Y = N and Z — {0, 1} and the program 
F, and define as before. t{iy(x) = 1 if and only if F 
terminates on input x. It is a classical fact of computability 
theory that the t^y function is not computable for all F [16] . 



end for 

p *— c/n 

With the same notations as in the previous sub-section: 



(c) 



< 



T w and thus the confidence interval is still valid: 



there is at least a 1 - 
Etw is less than p 1 



e probability that the true expectation 
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We shall see in the following section how to build abstract 
interpreters with a view to using them for this Monte-Carlo 
method. 

3 A CONCRETE SEMANTICS SUIT- 
ABLE FOR ANALYSIS 

From the previous section, it would seem that it is easy to 
use any abstract interpreter in a Monte-Carlo method. Alas, 
we shall now see that special precautions must be taken in 
the presence of calls to random generators inside loops or, 
more generally, fixpoints. 

3.1 Concrete Semantics 

We have for now spoken of deterministic programs taking 
one input x chosen according to some random distribution 
and one input y in some domain. Calls to random gener- 
ators (such as the POSIX drand48() function) are usually 
modeled by a sequence of independent random variables. If 
a bounded number of calls (< N) to such generators is used 
in the program, we can consider them as input values: x is 
then a tuple . . . , xn, v) where x\, . . . , x„ are the values 
for the generator and v is the input of the program. If an 
unbounded number of calls can be made, it is tempting to 
consider as an input a countable sequence of values (i„)„ 6 m 
where x\ is the result of the first call to the generator, X2 
the result of the second call. . . ; a formal description of such 
a semantics has been made by Kozen 8 9. 

Such a semantics is not very suitable for program analysis. 
Intuitively, analyzing such a semantics implies tracking the 
number of calls made to number generators. The problem is 
that such simple constructs as: 

if (...) { random () ; } else {} 

are difficult to handle: the countings are not synchronized 
in both branches. 

We shall now propose another semantics, identifying oc- 
currences of random generators by their program location 
and loop indices. The Backus-Naur form of the program- 
ming language we shall consider is: 



instruction 



elementary 

instruction ; instruction 
if boolean_expr 
then instruction 
else instruction 
end if 

while boolean_expr 
do instruction 
done 



We leave the subcomponents largely unspecified, as they 
are not relevant to our method, elementary instructions 



are deterministic, terminating basic program blocks like as- 
signments and simple expression evaluations. boolean_expr 
boolean expressions, such as comparisons, have semantics as 
sets of acceptable environments. For instance, a boolean_expr 
expression can be x < y + 4; its semantics is the set of exe- 
cution environments where variables x and y verify the above 
comparison. If we restrict ourselves to a finite number n of 
integer variables, an environment is just a n-tuple of integers. 

The denotational semantics of a code fragment c is a map- 
ping from the set X of possible execution environments be- 
fore the instruction into the set Y of possible environments 
after the instruction. Let us take an example. If we take 
environments as elements of Z 3 , representing the values of 
three integer variables x, y and z, then [x:=y+z] is the 
strict function (x, y, z) i— > (y + z,y,z). Semantics of basic 
constructs (assignments, arithmetic operators) can be easily 
dealt with this forward semantics; we shall now see how to 
deal with flow control. 

The semantics of a sequence is expressed by simple com- 
position 

[ei; ea] = lea] ° [ei] (1) 

Tests get expressed easily, using as the semantics [c] of a 
boolean expression c the set of environments it matches: 

[if c then ei else ea] (x) = 

if x G {a} then [ei] (x) else [e 3 ] (x) (2) 

and loops get the usual least-fixpoint semantics (considering 
the point-wise extension of the Scott flat ordering on partial 
functions) 

[while c do /] = lfp(\<j>.\x. 

if x G [c] then o [/] (x) else x). (3) 

Non-termination shall be noted by _L. 

As for expressions, the only constructs whose semantics 
we shall precise are the random generators. We shall con- 
sider a finite set G of different generators. Each generator g 
outputs a random variable r g with distribution /i 9 ; each call 
is independent from the precedent calls. Let us also con- 
sider the set P of program points and the set N* of finite 
sequences of positive integers. The set C = P x N* shall 
denote the possible times in an execution where a call to a 
random generator is made: (p, nxn2...ni) notes the execution 
of program point p at the ni-th execution of the outermost 
program loop, . . . , rij-th execution of the innermost loop at 
that point. C is countable. We shall suppose that inside the 
inputs of the program there is for each generator g in G a 
family (gip,w)){p,w)ec of random choices. 

The semantics of the language then become: 

[ei; e a ] = |ea] o [ei] (4) 

Tests get expressed easily, using as the semantics [c] of a 
boolean expression c the set of environments it matches: 

[if c then ei else ea].(w,a;) = 

if x G [cj then [ei] .{w, x) else [ea] - {w, x) (5) 

Loops get the usual least-fixpoint semantics (considering 
the point-wise extension of the Scott flat ordering on partial 



functions) : 

[while c do f}.(w ,x ) = 
lfp (\(f>.\{w, x).if x G [c] then (f> o S o [/] {w, x)) else x).{1.wq, 

where S.{c.w,x) = ((c + l).w,x). The only change is that 
we keep track of the iterations of the loop. 
As for random expressions, 

{p : randomj .{w, x) = g {p ^ w) (7) 

where p is the program point. 

This semantics is equivalent to the denotational semantics 
proposed by Kozen [SJO 2nd semantics] and Monniaux [11 J . 
the semantic of a program being a continuous linear opera- 
tor mapping an input measure to the corresponding output. 
The key point of this equivalence is that two invocations 
of random generators in the same execution have different 
indices, which implies that a fresh output of a random gen- 
erator is randomly independent of the environment coming 
to that program point. 

3.2 Analysis 

Our analysis algorithm is a randomized version of an ordi- 
nary abstract interpreter. Informally, we treat calls to ran- 
dom generators are treated as follows: 

• calls occurring outside fixpoint convergence iterations 
are interpreted as constants chosen randomly by the 
interpreter; 

• calls occurring inside fixpoint convergence iterations are 
interpreted as upper approximations of the whole do- 
main of values the random generator yield. 

For instance, in the following C program: 

int x; 

x = coin_flip() ; /* coin_flip() returns or 1 */ 

/* each with probability 0.5 */ 
for(i=0; i<5; i++) 
{ 

x = x + coin_flip(); 

} 

the first occurrence of coin_f lip() will be treated as a ran- 
dom value, while the second occurrence will be treated as 
the least upper bound of {0} and {1}. 

This holds for "naive" abstract interpreters; more ad- 
vanced ones might perform "dynamic loop unrolling" or 
other semantic transformations corresponding to a refine- 
ment of the abstract domain to handle execution traces: 



[while c do e] (x) = 




C 



(8) 

where ip(x) — [e] (x n [c]) and Ni and -/V2 are possibly de- 
cided at run-time, depending on the computed values. In 
this case, the interpreter uses a random generator for the 
occurrences of random g operations outside lfp computations 



and abstract values for the operations inside lfp's. Its exe- 
cution defines the finite set K of (p, m ...ni) tags uniquely 
identifying the random values chosen for g{ p> n 1 ...n,)j as wel l 
as the values (g c )c£K that have been chosen. This yields 

V(g c ) 9eG ,cec Vy € Y (Vc € K g c = g c ) 

l4{(9c) gG G,cec,y) € 7 z(«») (9) 

which means that 

V(ffc) 9 6G,c6C (Vc e K g c = g c ) ^ 

tw((gc)g£G,c£c) < rw{z ') (10) 

If we virtually choose randomly some t/ c for c ^ if, we know 
that tvi/((g c ) g gG iC gc) < Tjv(^ )• Furthermore, (g c ) follows 
the product random distribution [if c (each g c has been cho- 
sen independently of the others according to measure /j, g ). 

Let us summarize: we wish to generate upper bounds 
of experimental averages of a Bernoulli random variable 
tw '■ X — > {0, 1} whose domain has the product probability 
measure /ij ®^ g£G /if C where /ij is the input measure and 
the /i 9 's are the measures for the random number genera- 
tors. The problem is that the domain of this random vari- 
able is made of countable sequences; thus we cannot generate 
its input strictly speaking. We instead effectively choose at 
random a finite number of coordinates for the countable se- 
quences, and compute a common upper bound for tw for all 
inputs identical to our chosen values on this finite number 
of coordinates. This is identical to virtually choosing a ran- 
dom countable sequence x and getting an upper bound of its 
image by tw- 

Implementing such an analysis inside an ordinary abstract 
interpreter is easy. The calls to random generators are inter- 
preted as either a random generation, or as the least upper 
bound over the range of the generator, depending on a "ran- 
domize" flag. This flag is adjusted depending on whether 
the interpreter is computing a fixpoint. The interpreter does 
not track the indices of the random variables: these are only 
needed for the proof of correctness. The analyzer does a cer- 
tain number n of trials and outputs the experimental average 
ty^ . As a convenience, our implementation also outputs the 
iyy+t upper bound so that there is at least a probability 1— e 
that this upper bound is safe according to inequation 
This is the value that is reported in the experiments of sec- 
tion [5] 

While our explanations referred to a forward semantics, 
the abstract interpreter can of course combine forward and 
backward analysis [2] section 6] , provided the chosen random 
values are recorded so that subsequent passes of analysis 
can reuse them. Another related improvement, explained 
in section [4] uses a preliminary backward analysis prior to 
random generation. 

3.3 Arbitrary control- flow graphs 

The abstract interpretation framework can be extended to 
logic languages, functional languages and imperative lan- 
guages with recursion and other "complex programming 
mechanisms (call-by-reference, local procedures passed as 
parameters, non-local gotos, exceptions)" 1 . In such cases, 
the semantics of the program are expressed as a fixpoint of 



a system of equations over parts of the domain of environ- 
ments. The environment in that case includes the program 
counter, call stack and memory heap; of course a suitable 
abstract lattice must be used. 

Analyzing a program P written in a realistic imperative 
language is very similar to analyzing the following inter- 
preter: 

s <— initial state for P 

while s is not a termination state do 
s <- N(s) 

end while 

where N(s) is the next-state function for P (operational se- 
mantics). The abstract analyzer analysis that loop using an 
abstract state and an abstract version of N. Most analy- 
ses partition the domain of states according to the program 
counter, and the abstract interpreter then computes the least 
fixpoint of a system of semantic equations. 

Such an analysis can be randomized in exactly the same 
fashion as the one for block-structured programs presented 
in the previous section. It is all the same essential to store 
the generated values in a table so that backwards analysis 
can be used. 

4 COMPLEXITY 

The complexity of our method is the product of two inde- 
pendent factors: 

• the complexity of one ordinary static analysis of the 
program; strictly speaking, this complexity depends not 
only on the program but on the random choices made, 
but we can take a rough "average" estimate that de- 
pends only on the program being analyzed; 

• the number of iterations, that depends only on the re- 
quested confidence interval; the minimal number of it- 
erations to reach a certain confidence criterion can be 
derived from inequalities 18; appendix A] such as in- 
equation pip and does not depend on the actual pro- 
gram being analyzed. 

We shall now focus on the latter factor, as the former de- 
pends on the particular case of analysis being implemented. 

Let us recall inequation pip : Pr ^Etw > iyP + tj < 

e~ 2nt . It means that to get with 1 — e probability an ap- 
proximation of the requested probability fi, it is sufficient to 
compute an experimental average over ["— trials. 

This exponential improvement in quality (Fig. [1} is nev- 
ertheless not that interesting. Indeed, in practice, we might 
want £ and t of the same order of magnitude as /i. Let us 
take £ — at where a is fixed. We then have n ~ — ^§^, 
which indicates prohibitive computation times for low prob- 
ability events (Fig. [2}. This high cost of computation for 
low-probability events is not specific to our method; it is true 
of any Monte-Carlo method, since it is inherent in the speed 
of convergence of averages of identically distributed random 
variables; this relates to the speed of convergence in the cen- 
tral limit theorem |18l ch 1]. It can nevertheless be circum- 
vented by tricks aimed at estimating the desired low prob- 
ability by computing some other, bigger, probability from 
which the desired result can be computed. 
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Figure 1: Upper bound on the probability that the computed 
probability exceeds the real value by more than t, for t = 
0.01. 
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Figure 2: Numbers of iterations necessary to achieve a prob- 
ability of false report on the same order of magnitude as the 
error margin. 



int x , i ; 

know (x>=0 kk x<=2) ; 
i=0; 

while (i < 5) 
{ 

x += coin_f lip() ; 
i++; 

} 

know (x<3) ; 

Figure 3: Discrete probabilities. The analyzer es- 
tablishes that, with 99% safety, the probability p of 
the outcome (x < 3) is less than 0.509 given worst-case 
nondeterministic choices of the precondition (x > OAx < 2). 
The analyzer used n = 10000 random trials. Formally, p is 
Pr (coin_f lip e {0, l} 5 | 3x G [0, 2] n Z [PJ (coin.f lip, x) < 3) . 
Each coin_f lip is chosen randomly in {0, 1} with a uniform 
distribution. The exact value is 0.5. 

Fortunately, such an improvement is possible in our 
method. If we know that Tfi([cJ~ (W)) C 7?, with a mea- 
surable R, then we can replace the random variable tw by 
its restriction to R: tw\R\ then Etw = Pr (R) .Etw\n- If 
Pr (R) and Etw are on the same order of magnitude, this 
means that Etw\n will be large and thus that the number 
of required iterations will be low. Such a restricting R can 
be obtained by static analysis, using ordinary backwards ab- 
stract interpretation. 

A salient point of our method is that our Monte-Carlo 
computations are highly parallelizable, with linear speed- 
ups: n iterations on 1 machine can be replaced by n/m iter- 
ations on m machines, with very little communication. Our 
method thus seems especially adapted for clusters of low- 
cost PC with off-the-shelf communication hardware, or even 
more distributed forms of computing. Another improvement 
can be to compute bounds for several W sets simultaneously, 
doing common computations only once. 

5 PRACTICAL IMPLEMENTATION 
AND EXPERIMENTS 

We have a prototype implementation of our method, imple- 
mented on top of an ordinary abstract interpreter doing for- 
ward analysis using integer and real intervals. Figures [3] to [5] 
show various examples for which the probability could be 
computed exactly by symbolic integration. Figure [6] shows a 
simple program whose probability of outcome is difficult to 
figure out by hand. Of course, more complex programs can 
be handled, but the current lack of support of user-defined 
functions and mixed use of reals and integers prevents us 
from supplying real-life examples. We hope to overcome 
these limitations soon as implementation progresses. 

6 CONCLUSIONS 



We have proposed a generic method that combines the 
well-known techniques of abstract interpretation and Monte- 
Carlo program testing into an analysis scheme for probabilis- 
tic and nondeterministic programs, including reactive pro- 



double x; 

know (x>=0 . kk x<=l . ) ; 

x+=unif orm ( ) +unif orm ( ) +unif orm ( ) ; 

know (x<2. ) ; 

Figure 4: Continuous probabilities. The analyzer es- 
tablishes that, with 99% safety, the probability p of the 
outcome (a; < 2) is less than 0.848 given worst-case nonde- 
terministic choices of the precondition (x > A i < 1). 
The analyzer used n = 10000 random trials. Formally, 
p is Pr (uniform G [0, l] 3 | 3x G [0, 1] [P] (uniform, x) < 2) . 
Each uniform is chosen randomly in [0, 1] with the Lebesgue 
uniform distribution. 
The exact value is 5/6 « 0.833. 



double x , i ; 

know(x<0.0 kk x>0. 0-1.0); 
i=0.; 

while (i < 3.0) 
{ 

x += unif orm() ; 
i += 1.0; 

> 

know (x<1.0); 

Figure 5: Loops. The analyzer establishes that, with 
99% safety, the probability p of the outcome (x < 
1) is less than 0.859 given worst-case nondeterministic 
choices of the precondition (x < A x > —1). The 
analyzer used n — 10000 random trials. Formally, 
p is Pr (uniform G [0, l] 3 | 3x G [0, 1] [P] (uniform,^) < l) . 
Each uniform is chosen randomly in [0, 1] with the Lebesgue 
uniform distribution. The exact value is 5/6 w 0.833. 



{ 

double x, y, z; 

know (x>=0. kk x<=0.1); 

z=uniform() ; z+=z; 

if (x+z<2.) 

{ 

x += unif orm () ; 
} else 
{ 

x -= unif orm () ; 

> 

know (x>0.9 kk x<l.l) ; 

} 

Figure 6: The analyzer establishes that, with 99% safety, 
the probability p of the outcome (x > 0.9 A x < 1.1) 
is less than 0.225 given worst-case nondeterministic 
choices of the precondition (x > A x < 0.1). Formally, p is 
Pr (uniform G [0, l] 2 | 3x G [0,0.1] [P] (uniform, a;) G [0.9, 1.1]). 
Each uniform is chosen randomly in [0, 1] with the Lebesgue 
uniform distribution. 



grams whose inputs are modeled by both random and non- 
deterministic inputs. This method is mathematically proven 
correct, and uses no assumption apart from the distributions 
and nondeterminism domains supplied by the user. It yields 
upper bounds on the probability of outcomes of the program, 
according to the supplied random distributions, with worse- 
case behavior according to the nondeterminism; whether or 
not this bounds are sound is probabilistic, and a lower-bound 
of the soundness of those bounds is supplied. While our ex- 
planations are given using a simple imperative language as 
an example, the method is by no means restricted to imper- 
ative programming. 

The number of trials, and thus the complexity of the com- 
putation, depends on the desired precision. The method is 
parallelizable with linear speed-ups. The complexity of the 
analysis, or at least its part dealing with probabilities, in- 
creases if the probability to be evaluated is low. However, 
static analysis can come to help to reduce this complexity. 

We have implemented the method on top of a simple static 
analyzer and conducted experiments showing interesting re- 
sults on small programs written in an imperative language. 
As implementation progresses, we expect to have results on 
complex programs akin to those used in embedded systems. 
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A PROBABILITY THEORY 

Throughout this paper we take the usual mathematical point 
of view of considering probabilities to be given by measures 
over measurable sets |17l |4], 

• A u-algebra is a set of subsets of a set X that contains 
and is stable by countable union and complementa- 
tion (and thus contains X and is stable by countable 
intersection). For technical reasons, not all sets can be 
measured (that is, given a probability) and we have to 
restrict ourselves to some sufficiently large cr-algebras, 
such as the Borel or Lebesgue sets [17] . 

• A set X with a a- algebra ax defined on it is called a 
measurable space and the elements of the <r-algebra 
are the measurable subsets. We shall often men- 
tion measurable spaces by their name, omitting the a- 
algebra, if no confusion is possible. 



• If X and Y are measurable spaces, / : X — *• Y is a 
measurable function if for all W measurable in Y , 
f~ (W) i s measurable in X. 

• A positive measure is a function /i defined on a 
(j-algebra ax whose range is in [0, oo] and which is 
countably additive, /i is countably additive if, taking 
(^4n)neN a disjoint collection of elements of ax, then 
jit (Un=o-A n ) = IZ^Lq M-^n). To avoid trivialities, we 
assume (J,(A) < oo for at least one A. 

If X is countable, ax can be r P{X), the power-set of 
X, and a measure /i is determined by its value on the 
singletons: for any A<Z X, fi(A) = ~^2 aSA /i({a}). 

• A probability measure is a positive measure of total 
weight 1; a sub-probability measure has total weight 
less or equal to 1. We shall note M.<i(X) the sub- 
probability measures on X. 

• Given two sub-probability measures /i and // (or more 
generally, two <7-nnite measures) on X and X' respec- 
tively, we note <g) // the product measure [I7j defini- 
tion 7.7], defined on the product cr-algebra ax x a x >- 
The characterizing property of this product measure is 
that fx ® /jf(A x A') = fi(A).n' (A') for all measurable 
sets A and A'. It is also possible to define countable 
products of measures; if fi is a measure over the mea- 
surable space X, then //® N is a measure over the set X l% 
of sequences of elements of X. 

For instance, let us take fi the measure on the set {0, 1} 
with /x({l}) = p and ^i({0}) = 1 — p. Let us take S 
the set of sequences over {0,1} beginning with (0,0,1,0). 
/x® N (5') = p(l — p) 3 is the probability of getting a sequence 
beginning with (0, 0, 1, 0} when choosing at random a count- 
able sequence of {0, 1} independently distributed follow- 
ing ix. 

B ESTIMATING THE PROBABIL- 
ITY OF A RANDOM EVENT BY 
THE MONTE-CARLO METHOD 

We consider a system whose outcome (success or failure) 
depends on the value of a parameter x, chosen in the set X 
according to a random distribution fi. The behavior of this 
system is described by a random variable V : X —> {0, 1}, 
where means success and 1 failure. 

The law of large numbers says that if we indepen- 
dently choose inputs Xk, with distribution fj,, and com- 
pute the experimental average V'"' = — $^fc=i V(xk), then 
lim n ^oo V*-™' = EV where EV is the expectation of fail- 
ure. Intuitively, it is possible to estimate accurately EV by 
effectively computing V^ n ' for a large enough value of n. 

Just how far should we go? Unfortunately, a general fea- 
ture of all Monte-Carlo methods is their slow asymptotic 
convergence speed. Indeed, the distribution of the exper- 
imental average is a binomial distribution centered 
around EV . With large enough values of n (say n > 20), this 
binomial distribution behaves mostly like a normal (Gaus- 
sian) distribution (Fig. [7} with means p = EV and standard 
deviate ■ More generally, the central limit theorem 
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Figure 7: The Gaussian normal distribution centered on 0, 
with standard deviate 1. 

predicts that the average of n random variables identically 
distributed as V has the same expectation EV' as V and 
standard deviate -?= where a is the standard deviate of V. 
The standard deviate measures the error margin on the com- 
puted result: samples from a gaussian variable centered on 
xq and with standard deviate a fall within [xq — 2a, x + 2a] 
about 95% of the time. 

We can better evaluate the probability of underestimating 
the probability by more than t using the Chernoff-Hoeffding 
[6] [HI inequality A. 4. 4] bounds: 

Pr (EV > V (n) + 1) < e" 2 "* 2 (11) 

This bound, fully mathematically sound, means that the 
probability of underestimating V using by more than t 
is less than e~ 2nt . 

Any Monte-Carlo method has an inherent margin of error; 
this margin of error is probabilistic, in the sense that facts 
such as "the value we want to compute is in the interval 



[a, b]" are valid up to a certain probability. The size of the 
interval of safety for a given probability of error varies in 

C ABSTRACT INTERPRETATION 

Let us recall the mathematical foundations of abstract inter- 
pretation [3|[2]. Let us consider two preordered sets A" and 
Z* so that there exist monotone functions 7 a ■ A* — ■> 7(A), 
where A = X x Y, and -yw : Z* -> 9{Z), where 7(Z) is the 
set of parts of set Z, ordered by inclusion, 'yw is called the 
concretization function. 

The elements in A" and Z* represent some properties; 
for instance, if X = 7L m and Y = Z", A 8 could be the 
set of machine descriptions of polyhedra in Z m+n and 7a 
the function mapping the description to the set of points 
inside the polyhedron [3]. A simpler case is the intervals, 
where the machine description is an array of integer couples 
Gin i On } and its concretization is the set of 
tuples (ci; . . . ; c n ) where for all i, a% < a < hi. 

We then define an abstract interpretation of program 
c to be a monotone function [c] : A' —* Z" so that 

Va" G A\ Va G A a G 7A (A tt ) => [cj (a) G ~fz o {of (a 8 ). 

In the case of intervals, abstract interpretation propagates 
intervals of variation along the computation. Loops get a fix- 
point semantics: the abstract interpreter heuristically tries 
to find intervals that are invariant for the body of the loop. 
Such heuristics are based on widening and narrowing oper- 
ators 

It is all the same possible to define backwards abstract 
interpretation: a backwards abstract interpretation of 

a program c is a monotone function [cj^ 1 " : — » A" so 
that 

W» G Z\ Vz G Z z G 7 z(^ 8 ) => W X (z) C 7 A o [cr l8 ( 2 «). 

Further refinement can be achieved by iterating forwards and 
backwards abstract interpretations [2]. 



